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METHOD AND APPAE^IlTUS FOR USER-CONTROLLED SELECTIVE 
OVERLAY IN A STREAMING MEDIA 

CROSS REFERENCE TO RELATED APPLICATIONS 



The present invention is related to applications 
entitled METHOD AND APPARATUS FOR DYNAMIC DISTRIBUTION OF 
CONTROLLED AND ADDITIONAL SELECTIVE OVERLAYS IN A 
STREAMING MEDIA, serial no. , attorney docket 

10 no. AT 9- 99-2 50; and METHOD AND APPARATUS FOR DYNAMIC 
DISTRIBUTION OF CONTROLLED AND ADDITIONAL SELECTIVE 

OVERLAYS IN A STREAMING MEDIA, serial no. , 

attorney docket no. AT9-99-255; all of which are filed 
even date hereof, assigned to the same assignee, and 

15 incorporated herein by reference. 

BACKGROUND OF THE INVENTION 

20 1. Technical Field: 

The present invention relates to the field of 
computer software and, more particularly, to methods of 
providing streaming media to users. 

25 2. Description of Related Art: 

Internet, also referred to as an "internetwork", in 
communications is a set of computer networks, possibly 
dissimilar, joined together by means of gateways that 
handle data transfer and the conversion of messages from 
30 the sending network to the protocols used by the receiving 
network (with packets if necessary) • When capitalized, 
the term "Internet" refers to the collection of networks 
and gateways that use the TCP/IP suite of protocols. 

The Internet has become a cultural fixture as a 
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source of both information and entertainment. Many 
businesses are creating Internet sites as an integral part 
of their marketing efforts, informing consumers of the 
products or services offered by the business or providing 
5 other information seeking to engender brand loyalty. Many 
federal, state, and local government agencies are also 
employing Internet sites for informational purposes, 
particularly agencies which must interact with virtually 
all segments of society such as the Internal Revenue 
10 Service and secretaries of state. Operating costs may be 
reduced by providing informational guides and/or 
searchable databases of public records online. 

Furthermore, the Internet has destroyed traditional 
geographic and technical barriers to distribution of 
15 multimedia content. Traditionally, multimedia content has 
been distributed via broadcast television (sometimes in 
combination with radio), moving pictures (film), and, more 
recently, cable television. Much of this multimedia 
content was only available within limited geographic areas 
20 and presented in one language. 

To reach wider audiences and provide consumers with 
more choices, television broadcasts have employed SAP 
broadcasting in which a television broadcast contained 
audio tracks for several different languages for the same 
25 video content. However, SAP is an all or nothing choice. 
That is, a user must choose one audio track. The 
possibility of mixing several audio tracks together is 
non-existent using SAP. 

Another method television broadcasters have used to 
30 provide users with more choices over the presentation of 
programming is "simulcast". With this technology, a 
different audio track was broadcast via radio to 
correspond to a simultaneously broadcast television 
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program. However, this involved the use of two different 
media. Furthermore, only the whole of the audio sound 
track was broadcast, not pieces of it which could be 
selectively filtered. Additionally, this method did not 

5 contain any other video components and there was not a 
single point of control. ^'Simulcast" required two points 
of control: a television and a separate radio. 

Currently, the potential is available for anyone 
anywhere in the world to view content being broadcast from 

10 anywhere in the world. This potential needs to be 

realized and requires a new technology to reduce overall 
bandwidth while providing viewer centric content as a way 
of differentiating a broadcaster from his competition in 
the world market. Also, to encourage viewers to watch 

15 an event or program it is desirable to appeal to as many 
different types of viewers as possible. In some cases, 
events or programs may be broadcast in different 
languages. This type of customization typically requires 
a separate broadcast for each language. This type of 

20 customization requires bandwidth for the audio and video 
for each version. As a result, bandwidth is wasted by 
rebroadcasting an event for each versions. Thus, it 
would be advantageous to have an improved method and 
apparatus to allow customization of the presentation of 

25 events to fit the viewing patterns, wants, and needs of 
present day consumers and reduce bandwidth needed to 
customize events. 
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SUMMARY OF THE INVENTION 

The present invention provides a method and apparatus for 
tailoring a multimedia presentation of an event on a 
5 computerized multimedia system. A set of video streams, a 
set of audio streams and a set of information streams are 
provided for the event via a network coupled to the 
computerized multimedia system. Video streams are 
selected for presentation from the set of video streams 
10 for the event. Audio streams are selected for 

presentation from the set of audio streams for the event. 
Information streams are selected for presentation from 
the set of information streams for the event. Responsive 
to user input, selected video streams and the selected 
15 audio streams are assigned to respective portions of 

video and audio output devices. The event is presented 
according to selected video stream and audio stream 
assignments . 
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BRIEF DESCRIPTION OF THE DRAWINGS 



The novel features believed characteristic of the 
invention are set forth in the appended claims. The 
invention itself, however, as well as a preferred mode of 
use, further objectives and advantages thereof, will best 
be understood by reference to the following detailed 
description of an illustrative embodiment when read in 
conjunction with the accompanying drawings, wherein: 

Figure 1 depicts a pictorial representation of a 
distributed data processing system in which the present 
invention may be implemented; 

Figure 2 depicts a block diagram of a data 
processing system which may be implemented as a server in 
accordance with the present inventions- 
Figure 3 depicts a block diagram of a data 
processing system in which the present invention may be 
implemented; 

Figure 4 shows a block diagram broadly illustrating 
the process of the present invention; 

Figure 5 is a block diagram of functional components 
used in the selection of media streams depicted in 
accordance with the present invention; 

Figure 6 is a diagram illustrating an example 
display depicted in accordance with a preferred 
embodiment of the present invention; 

Figures 7A-7D are diagrams illustrating data 
structures contained within data streams sent to a client 
depicted in accordance to a preferred embodiment of the 
present invention; 

Figure 8 is a flowchart of a process used to process 
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data streams depicted in accordance with a preferred 
embodiment of the present invention; 

Figure 9 is a flowchart of a process for generating 
CRC values for use in synchronizing data streams depicted 
in accordance with a preferred embodiment of the present 
invention; and 

Figure 10 is a flowchart of a process for 
synchronizing data streams using CRC data depicted in 
accordance with a preferred embodiment of the present 
invention. 



Docket No. AT9-99-254 



DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

With reference now to the figures, and in particular 

5 with reference to Figure 1, a pictorial representation of 
a distributed data processing system is depicted in which 
the present invention may be implemented. 

Distributed data processing system 100 contains 
network 102, which is the medium used to provide 

10 communications links between various devices and 

computers connected within distributed data processing 
system 100. Network 102 may include permanent 
connections, such as wire or fiber optic cables, or 
temporary connections made through telephone connections. 

15 In the depicted example, server system 104 is 

connected to network 102, along with storage unit 106. 
Server system 104 may contain one or more server 
computers for providing data streaming. For example, a 
server may be used to administer and control data streams 

20 while another server within server system 104 is employed 
to originate resource data streams to clients. The 
information in these data streams may be stored within 
server system 104 or on a remote storage device, such as 
storage device 106. In addition, clients 108, 110 and 

25 112 are also connected to network 102. These clients, 
108, 110 and 112, may be, for example, without 
limitation, personal computers, set top boxes, web TV 
units, or any other hardware unit for receiving data 
streams. Clients 108, 110 and 112 are clients to server 

30 system 104. In the depicted example, server system 104 
provides data streams to clients 108-112. Distributed 
data processing system 100 may include additional 
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servers, clients, and other devices not shown. 

Clients 108, 110, and 112 include mechanisms for 
mixing and filtering data streams received from server 
system 104 or other server systems not shown. These 
5 mechanisms may be implemented in hardware, software, 
firmware, or some combination thereof in the depicted 
examples . 

Distributed data processing system 100 may be, for 
example, the Internet, with network 102 representing a 
10 worldwide collection of networks and gateways that use 
the TCP/IP suite of protocols to communicate with one 
another. At the heart of the Internet is a backbone of 
high-speed data communication lines between major nodes 
or host computers consisting of thousands of commercial, 
15 government, education, and other computer systems that 
route data and messages. 

Another implementation for network 102 is as a 
broadband network, also referred to as a broadband 
integrated services digital network (BISDN) . In such a 
20 network, broadband services typically provide channels 
capable of supporting data transmission rates of greater 
than 1.5Mbps or a primary rate in integrated services 
digital network (ISDN), Tl, or DSl. In broadband system, 
the type of transfer mode typically is a synchronous 
25 transfer mode (ATM) with the use of synchronous optical 
network (SONET) and intelligent network (IN) technologies 
as providing the framework for ISDN. In such a network, 
information including voice, data, video, and audio are 
transferred through the network to various clients. 
30 Further, network 102 also may be a cable 

communications system depending on the implementation. Of 
course, distributed data processing system 100 also may 
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be implemented as a number of different types of networks 
such as, for example, an intranet or a local area 
network. Figure 1 is intended as an example and not as 
an architectural limitation for the processes of the 
5 present invention. 

Referring to Figure 2, a block diagram of a data 
processing system, which may be implemented as a server 
■ system, such as server system 104 in Figure 1, is 

depicted in accordance with the present invention. Data 
10 processing system 200 may be employed as a video server 
or other application server within a server system. 

Data processing system 200 may be a symmetric 
multiprocessor (SMP) system including a plurality of 
processors 202 and 204 connected to system bus 206. 
15 Alternatively, a single processor system may be employed. 
Also connected to system bus 206 is memory 
controller/cache 208, which provides an interface to 
local memory 209. I/O bus bridge 210 is connected to 
system bus 206 and provides an interface to I/O bus 212. 
20 Memory controller/cache 208 and I/O bus bridge 210 may be 
integrated as depicted. 

Peripheral component interconnect (PCI) bus bridge 
214 connected to I/O bus 212 provides an interface to PCI 
local bus 216. A number of modems 218-220 may be 
25 connected to PCI bus 216. Typical PCI bus 

implementations will support four PCI expansion slots or 
add-in connectors. Communications links to network 
computers 108-112 in Figure 1 may be provided through 
modem 218 and network adapter 220 connected to PCI local 
30 bus 216 through add-in boards. 

Additional PCI bus bridges 222 and 224 provide 
interfaces for additional PCI buses 226 and 228, from 



10 

Docket No. AT9'-99-254 

which additional modems or network adapters may be 
supported, in this manner, server 200 allows connections 
to multiple network computers. A memory mapped graphics 
adapter 230 and hard disk 232 may also be connected to 
I/O bus 212 as depicted, either directly or indirectly. 

Those of ordinary skill in the art will appreciate 
that the hardware depicted in Figure 2 may vary. For 
example, other peripheral devices, such as optical disk 
drives and the like, also may be used in addition to or 
in place of the hardware depicted. The depicted example 
is not meant to imply architectural limitations with 
respect to the present invention. 

The data processing system depicted in Figure 2 may 
be, for example, an IBM RS/6000, a product of 
International Business Machines Corporation in Armonk, 
New York, running the Advanced Interactive Executive 
(AIX) operating system. 

With reference now to Figure 3, a block diagram of a 
data processing system in which the present invention may 
be implemented is illustrated. Data processing system 
300 is an example of a computer. Data processing system 
300 employs a peripheral component interconnect (PCI) 
local bus architecture. Although the depicted example 
employs a PCI bus, other bus architectures, such as Micro 
Channel and ISA, may be used. Processor 302 and main 
memory 304 are connected to PCI local bus 306 through PCI 
bridge 308. PCI bridge 308 may also include an 
integrated memory controller and cache memory for 
processor 302. Additional connections to PCI local bus 
306 may be made through direct component interconnection 
or through add-in boards. In the depicted example, local 
area network (LAN) adapter 310, SCSI host bus adapter 
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312, and expansion bus interface 314 are connected to PCI 
local bus 306 by direct component connection. In 
contrast, audio adapter 316, graphics adapter 318, and 
audio/video adapter (A/V) 319 are connected to PCI local 
5 bus 306 by add-in boards inserted into expansion slots. 
Expansion bus interface 314 provides a connection for a 
keyboard and mouse adapter 320, modem 322, and additional 
memory 324. In the depicted example, SCSI host bus 
adapter 312 provides a connection for hard disk drive 
10 326, tape drive 328, CD-ROM drive 330, and digital video 
n disc read only memory drive (DVD-ROM) 332. Typical PCI 

'2 local bus implementations will support three or four PCI 

15 expansion slots or add-in connectors. 

;2 An operating system runs on processor 302 and is 

Ifl 15 used to coordinate and provide control of various 
•P components within data processing system 300 in Figure 3. 

Q The operating system may be a commercially available 

;° operating system, such as OS/2, which is available from 

Q International Business Machines Corporation. "OS/2" is a 

!S 20 trademark of International Business Machines Corporation. 

An object oriented programming system, such as Java, may 
run in conjunction with the operating system, providing 
calls to the operating system from Java programs or 
applications executing on data processing system 300. 
25 Instructions for the operating system, the 

object-oriented operating system, and applications or 
programs are located on a storage device, such as hard 
disk drive 326, and may be loaded into main memory 304 
for execution by processor 302. 
30 Those of ordinary skill in the art will appreciate 

that the hardware in Figure 3 may vary depending on the 
implementation. For example, other peripheral devices. 



12 

Docket No. AT9^99-254 

such as optical disk drives and the like, may be used in 
addition to or in place of the hardware depicted in 
Figure 3. The depicted example is not meant to imply 
architectural limitations with respect to the present 
5 invention. For example, the processes of the present 
invention may be applied to multiprocessor data 
processing systems. Additionally, data processing system 
300 may take the form of a notebook computer or a 
personal digital assistant (PDA) . Further, the data 
10 processing system may be, for example, without 

limitations, a television or settop cable box containing 
appropriate hardware and software for implementing 
processes of the present invention. 

The present invention provides a method, apparatus, 
15 and computer implemented instructions for decreasing 
bandwidth usage and for providing increased client 
customization of programs or events. The mechanism of 
the present invention is directed towards tailoring a 
multimedia presentation on a multimedia data processing 
20 system. This is accomplished by breaking up a program or 
event into different data streams for different audio and 
video components. Additional optional data streams may 
be added to allow for customization of the program or 
event. A data processing system may receive a set of 
25 video and audio streams associated with a program or 

event. These data streams may be selectively presented 
based on user input. Depending on the user input, some 
portions of the video and audio data streams may be 
presented while others are omitted. Further, user input 
30 may selectively place a video stream on a display. Also, 
information streams containing text and/or graphics may 
be received and selectively displayed in association with 
the presentation of the program or event. The 
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combination of these data streams at the user site 
provides for customization without requiring broadcasting 
of multiple versions of an event to viewers. 

By allowing a viewer to customize a program or 
5 event, bandwidth usage may be reduced because program 

originators do not have to broadcast multiple versions of 
a program or event for different geographical regions or 
different viewer basis. 

Turning now to Figure 4, there is shown a block 
10 diagram broadly illustrating the process of the present 
invention. A client 400, such as data processing system 
300 in Figure 3, sends a request 402 for programming 
content to an originating server 404. Originating server 
404 is a data processing system, such as data processing 
15 system 200 in Figure 2, and is the host for the 

programming content requested by client 400. Originating 
server 404 sends the requested programming as a set of 
data streams 406 back to client 400. Data streams 406, 
may be used to present programming or events to a user at 
20 client 402. Furthermore, data streams 406 may comprise 
one or more video stream components, one or more 
information stream components, and one or more audio 
stream components. 

Client 400 and originating server 404 have been 
25 described and illustrated in Figure 4 as though they are 
directly connected. However, client 402 and originating 
server 404 are preferably part of a distributed data 
processing system such as distributed data processing 
system 100 and are linked together via a network, such as 
30 network 102 in Figure 1. 

Turning next to Figure 5, a block diagram of 
functional components used in the selection of media 
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streams in accordance with the present invention is 
depicted, in this example, client 500 includes a data 
stream processing system 502 to receive data streams 510 
from a network 504 for output to output system 506. The 
5 selection of data streams received from network 504 are 
controlled through user control 508, which controls data 
stream processing system 502. 

Multiple data streams may be received from network 
504 through connection 508. These data streams may be, 
10 for example, audio, video, text, graphics, or other 

information that may be displayed through output system 
506 to a user of client 500. Selection filtering of data 
streams from network 504 by data stream processing system 
502 are controlled through user control 508. The 
15 selected data stream or selected portions of data streams 
are displayed to user through output system 506, which 
may be used to drive displays and audio output devices, 
such as a high definition television or speakers. 
With reference now to Figure 6, a diagram 
20 illustrating an example display is depicted in accordance 
with a preferred embodiment of the present invention. In 
this example, a presentation system 600 is presented with 
a display unit 602, audio sources 604, 606, and 608. 
Presentation system 600 is the part of a client through 
25 which the user sees and hears programs. In this example, 
audio source 604 is a left speaker while audio source 608 
is a right speaker. Audio source 606 is the center 
speaker in this example. The video 610 is displayed in 
display 602 with text box 612, text box 614, and video 
30 616. Video 610 is the primary data stream in this 

example with text box 612 and text box 614 being overlaid 
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over video 610. 

Text box 612 and text box 614 are generated from 
two data streams in this example. Text box 612 is a 
stationary text box, which may be used to provide 
information, such as statistics about the program, 
upcoming programs, or any other information the user may 
select. Text box 614 in this example is a ^^pop-up," 
which may be used to provide information about a 
particular object or person in video 610. 

Additionally, the data packets in text box 614 may 
include data to reposition text box 614 to follow a 
particular object or person. Video 616 is displayed 
using another data stream to provide additional viewing 
for the user. For example, video 616 may provide 
different perspectives in the program or may illustrate 
scene not currently displayed within video 610. 
Initially, the user may select various audio streams for 
feed through 604, 606, and 608. For example, if the 
programming being displayed is a sporting event, the user 
might select a particular player and have audio 
information about that player played over 608 or have 
direct feeds from a microphone located on the player 
played through 604 . 

By way of example, consider a football game being 
broadcast across the Internet or a broadband network. 
The end user is able to select from and mix several audio 
streams. For example, the quarterback microphone of 
either team, the coach's microphone of either team, 
and/or the announcers, of which, there could be several 
30 (e.g., John Madden, Phil Simms, Howie Long, Bruce Beadle, 
Michael Paolini, etc.). In this manner, the user is able 
to tune-out (deselect) John Madden if they find him 
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offensive. The user might instead choose to listen to 
the Quarterback microphone of the 49er's (or the opposing 
team, or another announcer of their choice, or to listen 
in another language entirely) without having to change 
5 the channel /coverage. 

Carrying the football example further, with the 
present invention, it is possible to have a separate 
audio track for viewers that are new to the sport. This 
separate audio track would explain the rules of the game 
10 and what is happening in additional detail. It would 

also explain the technical jargon particular to football. 
The user may choose to have this additional information 
presented as a text stream. 

In yet another example of the use of the present 
15 invention, continuing with the football theme, the user 
could place team A' s player voices on the right channels 
with team B' s player voices on the left channels and the 
announcers' voices on the rear channels. Furthermore, 
the user could select the player with the ball such that 
20 this player's voice is always on the center channel. 

One primary data stream representing the video 
(which does not change no matter which audio overlay 
streams are selected) , and many small add-on data streams 
representing audio or video overlays may be used to 
25 present the program. These streams could be mixed 
together or assigned to channels (right, left, rear, 
center, etc.) and/or 3-D positional locations using the 
latest "Dolby Digital" from Dolby Laboratories and other 
similar technologies. In addition, the volume could be 
30 adjusted on a per-overlay basis. More information on 
technologies, such as "Dolby Digital" may be found in 
United States Patent No. 5,912,976. 

With respect to video overlays, several video 
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overlays can also be streamed and selected in a manner 
similar to the audio streaming and mixed to ^^overlay the 
final video. If, for example, the desired overlay was a 
balloon Shaped pop-up at a certain video display position 
5 containing some text, then the additional overlay stream 
would not have to be video itself, but rather the 
information to create the overlay video. Therefore, the 
size of the data stream would be much, much smaller than 
a primary video stream. Additionally, unrelated 
10 information in addition to the data streams selected for 
an event, such as, for example, stock market data or news 
headlines also may be provided to the user. If, for 
example, the event or programming being viewed is that of 
a football game, it is possible to have a video overlay 
15 stream dedicated to displaying pop-up balloons with data, 
such as statistics on players in which these pop-up 
balloons may appear over the players themselves. 
Additionally, betting odds, other interesting facts about 
the players, or rules and information about events may be 
20 displayed in this manner. Another type of data stream 

may include embedded pictures of players with biographies 

tagged on as well. 

For other types of programming, other pop-up windows 
may be provided. For example, for a murder mystery show, 
25 pop-up windows giving clues about future events may be 
displayed or facts about actors, the script, or director 
also may be displayed in association with the program. 
In the case of a murder mystery show, different levels of 
clues may be provided depending the selection made by the 
30 user. 

In this manner, in the context of the football game 
example, it is possible to have video overlay streams 
dedicated to displaying pop-up balloon (s) with data such 
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tatistics on players (they very well might appear 
the players themselves), betting odds, odd or 
Interesting facts, information explaining what is 
happening and rules (for novices), a suimary of different 
on-going sporting event data (such as scores and 
statistics from other games in progress), stock market 
data, and news headlines to provide but a few examples. 
Another stream type might contain embedded pictures of 
the players with bios tagged on as well. 

In another embodiment, it is possible to make the 
overlays selectable. That is to say that the user might 
do the equivalent of ^^clicking" on one, to trigger 
additional overlay streams, or lock it in place. For 
example, selecting a player's "name overlay" might for 
15 example bring up an overlay with that player's biography. 
In a one-way system, this could be done by having an 
overlay channel which continually cycles through all the 
players biographies, and having the client "select" the 
one of interest at the time of broadcast. 
20 Additionally, the user could select between the 

different options using their client and thus have a lot 
more control of the information they would like to see 
and hear during a broadcast. 

Turning now to Figures 7A-7D, diagrams illustrating 
data structures contained within data streams sent to a 
client are depicted in accordance with a preferred 
embodiment of the present invention. In Figure 7A, a 
data packet 700 is shown in which the data packet 
includes a header 702 and a payload 704. Header 702 
30 contains information used in routing and handling data 
contained within payload 704 of data packet 700. 

In Figure 7B, an example of header information found 



25 




19 

Docket NO. AT9~99-254 

within a .ata pac.et, such as data pac.et 700, is shown 
in this example, header 706 includes an ID f.eld 708, 
Channel field 710, a position field 712, and a volume 
field 714. 

5 in this example, header 706 is a header identifying 

an audio packet. ID field 708 is used to identify the 
type of data packet. Channel field 710 in this example 
may be used to identify the channel or channels in which 
the data packet is to be presented to the user. For 
10 example, channel field 710 may be used to identify 
channels such as those in a surround system. The 
channels may be, in this example, right, left, rear, or 
center. Additionally, position field 712 also is 
included such that three dimensional positional locations 
15 using sound technologies such as Dolby Digital may be 
employed. This positional information may provide a 
default positioning that may be altered by a user. Volume 
field 714 is included to allow for adjusting the volume 
of the data in this data packet relative to other data 
20 packets to allow for overlays or emphasis for different 
data streams. Other fields may be used in addition to or 
in place of fields shown in header 706 for presenting 
audio information. 

In Figure 7C, a header 716 is an example of header 
25 information found in a video packet used in a video 

stream. In this example, header 716 includes an ID field 
718, a time stamp field 720, and a CRC field 722. ID 
field 718 is used to identify the type of data contained 
within the packet, while time stamp 720 in this example 
30 is used for synchronization purposes. CRC field 722 may 
be used for error checking or synchronization purposes 
depending upon the implementation. If CRC field 722 is 
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used for synchronization purposes, the CRC data is 
calculated for another data stream and placed within this 
field prior to being transmitted or broadcast to a 
viewer. The CRC data may be, for example, calculated for 
a frame or a portion of a frame in the other data stream. 
When this data stream and the other data stream are 
received, CRC data is calculated for a frame in the other 
data stream. This calculated CRC data is compared with 
the CRC data stored in CRC field 722. Matches between 
0 these CRC values are used to identify synchronization 
points between the two data streams. 

Next, in Figure 7D, another example of a header used 
for data packets in a video stream is illustrated. 
Header 724 includes an ID field 726, a type field 728, a 
5 location field 730, a duration field 732, and a z depth 
field 734. ID field 726 is used to identify the data 
packet as containing video data. Type field 728 in this 
example may be used to identify the type of video, such 
as text, or a frame in a moving video stream. Location 
20 field 730 may be used to identify the location on the 
screen at which the data is to be displayed. Duration 
field 732 may identify the amount of time the data is to 
be displayed. Z depth field 734 may be used to identify 
the depth or order in which the data is to be displayed 
25 relative to other video streams. 

In the example, header 724 is an example of a header 
used for a video stream containing text to be displayed 
as a balloon or pop-up on the screen. In this case, 
duration field 732 identifies how long the text is to be 
30 displayed while the z depth field 734 identifies where 
the text is to be located with respect to other video 
streams. The z depth is used to determine the order in 
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which video streams or overlays are displayed on the 
display. Location field 730 provides the xy location on 
the screen. In the text for display in the pop-up would 
be found in the payload. In this manner, a video overlay 
5 stream dedicated to displaying text, may be used along 
with moving video data streams. 

The present invention also provides for 
selectability of these different data streams. For 
example, user may be able to press a selected button to 
10 trigger additional overlay streams or to lock one in 

place. For example, selecting a player's "name overlay" 
might bring up an overlay with that player's biography. 
In a one way system, this may be performed by having a 
overlay channel which continually cycles through all of 
15 the players' biographies and having the client machine 
select the one of interest at the time of broadcast. In 
this manner, user may select between different options 
and have more control on the information that the user 
desires to see and hear during a program. The selections 
20 may be implemented via a one way or two way communication 
depending on the bandwidth and type of network in use. 

Overlay streams may be controlled from the server. 
In such a case, a command may be imbedded in an overlay 
or a set of commands may be imbedded to cause overlays to 
25 be selected or deselected on the client. Such a 

mechanism would allow for switching selected overlays off 
to broadcast an important event, and then resuming the 
previous programming. 

Turning now to Figure 8, a flowchart of a process 
30 used to process data streams is depicted in accordance 

with a preferred embodiment of the present invention. To 
start, a user selects programming to be retrieved from a 
distributed database (step 800) . The user receives 
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Itiple video and audio streams corresponding to the 
selected program (step 802) . The user selects audio 
streams (step 804), either directly in real time or 
through the use of user defined configurations, and, if 
raore than one audio stream is selected (step 806), then 
the user configures the presentation profile for the 
selected audio streams (step 808) . For example, if three 
audio streams are chosen, the user could configure the 
system to play one audio stream on the left channel, one 
audio stream on a center channel, and one audio stream on 
a right channel. Alternatively, the user could configure 
the presentation of the selected audio streams such that 
the volume of one audio stream is louder than the volume 
of the other two audio streams. 

Next, the user selects, from the plurality of video 
streams received, one or more video streams to be 
displayed (step 811) . If more than one video stream is 
selected (step 814), then the user configures the 
presentation of the multiple video streams (step 816) . 
20 For example, one video stream could be presented on the 
left side of the video display and a second video stream 
could be presented on the right side of the video 
display. In another example, one video stream might 
occupy the entire video display screen, while a second 
25 video stream was placed in a smaller window within the 

first video stream. Also, video streams are configurable 
for such things as opacity and scaling, as well as other 
factors. Opacity is the semi-transparency as seen in 
menus overlaid on an event in which the event can be seen 
30 through the menu. Scaling may be employed to size 
display of a video stream on a display device. 

Once the media streams are selected and configured 
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by the user, the audio and video streams are synchronized 
to each other (step 818) and presented to the user in a 
user configured manner (step 820) . 

With reference again to step 806, if more than one 
5 audio stream is not selected, then the user selects, from 
the plurality of video streams received, one or more 
video streams to be displayed (step 811) . 

With reference again to step 814, if more than one 
video stream is not selected, the audio and video streams 
10 are synchronized to each other (step 818) and presented 
to the user in a user configured manner (step 820) . 
Several mechanisms may be employed to synchronize the 
data streams. For example, a time stamp may be sent with 
each packet of data in each media stream, then each 
15 selected data stream is synchronized to be presented to 
the user such that data packets with identical time 
stamps are presented at the same time. 

Each data stream also may contain a spike in the 
data that appears at a periodic rate. The spike in the 
20 data may be, for example, a data packet that contains no 
video or audio. Further, each data stream contains a 
larger spike that appears at a longer periodic rate. For 
example, the shorter spike may appear every 30 frames 
wherein the larger spike appears every 10 seconds. The 
25 selected data streams may be synchronized such that they 
are presented to the user such that the spikes appear in 
the data streams simultaneously. The longer spike 
enables the system to resynchronize itself periodically 
if the data streams have become separated by a period of 
30 one or two smaller data spikes. 

In a third method of synchronization, cyclic 
redundancy check (CRC) values. Synchronizing data streams 
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using a CRC data Involves calculating CRC data for a 
first data stream and placing those calculated values in 
the headers of data packets in the second data stream 
The CRC values are placed in data packets in the second 
5 data stream that should be synchronized with data packets 
in the first data stream. 

Turning to Figure 9, a flowchart of a process for 
generating CRC values for use in synchronizing data 
streams is depicted in accordance with a preferred 
10 embodiment of the present invention. The process 

illustrated is used for video streams, but may be applied 
to other types of data streams. 

The process begins by selecting an unprocessed frame 
(step 900) . This step selects the first unprocessed 
15 frame for content that will be transmitted as a data 
stream. A CRC value is then calculated for the frame 
(step 902) . The CRC value may be calculated for an 
entire frame or different portions of the frame. For 
example, the CRC value may be based on the first five 
20 bytes of the frame, five bytes in the middle of the 
frame, and the last five bytes in the frame. 

Next, the portion of a second data stream that is to 
be synchronized with this frame is identified (step 904). 
The CRC value is then placed into the header of a data 
25 packet for that portion of the data stream (step 906) . A 
determination is then made as to whether additional 
unprocessed frames are present for processing (step 908). 
If additional unprocessed frames are present, the process 
returns to step 900. Otherwise, the process terminates. 
30 The process in Figure 9 may be used when the data 

streams are transmitted or broadcast. Alternatively, 
these processes may be initiated prior to transmitting or 
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broadcasting a data stream. In such a case, the CRC 
values may be stored and added to the second data stream 
when that data stream is transmitted. 

Turning now to Figure 10, a flowchart of a process 
5 for synchronizing data streams using CRC data is depicted 
in accordance with a preferred embodiment of the present 
invention. The processes illustrated in Figure 9 may be 
implemented at the viewer site to synchronize incoming 
data streams for presentation of a program or event. 
10 The process begins by receiving data streams (step 

1000) . In step 1000, the data streams are received for 
presentation. These data streams are placed in a buffer 
prior to presentation (step 1002) . A CRC value is 
calculated for a frame in the first data stream stored in 
15 the buffer (step 1004) . Thereafter, the calculated CRC 
value is compared to CRC values in the second data stream 
stored in the buffer to identify a point in the second 
data stream containing a matching CRC value (step 1006) . 
In the depicted examples, CRC values are placed within 
20 headers in the second data stream at points where the CRC 
value should match a calculated CRC value from a portion 
of the first data stream. Thereafter, the first and 
second data stream are presented at the matching or 
synchronization points (step 1008) with the process then 
25 returning to step 1000. The present invention could be 
implemented via one-way or two-way communication 
depending on the bandwidth of the network type in use. 

Thus, the present invention provides a customized 
experience for a user. The present invention provides 
30 this customized experience by providing a number of 

different types of data streams, audio and video, which 
are delivered to a client in which the client may select 
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frora these data streams to customize the program or event 
.eing Viewed by the user. Further, the present xnvent.on 
allows for decreasing the amount of bandwidth needed to 
provide customized programing. Currently, a data stream 
is created for each version or customization of a program 
transmitted to viewers in which the audio and video 
cannot be selectively modified. This advantage is 
provided by sending a set of data streams from which 
different users may select a subset of the data streams 
for customizing a program or event, rather than creating 
a new data stream in which audio and video are 
transmitted in an unmodifiable form for each desired 
customization of a program or event. 

It is important to note that while the present 
invention has been described in the context of a fully 
functioning data processing system, those of ordinary 
skill in the art will appreciate that the processes of 
the present invention are capable of being distributed in 
the form of a computer readable medium of instructions 
and a variety of forms and that the present invention 
applies equally regardless of the particular type of 
signal bearing media actually used to carry out the 
distribution. Examples of computer readable media 
include recordable-type media such a floppy disc, a hard 
disk drive, a RAM, and CD-ROMs and transmission-type 
media such as digital and analog communications links. 

The description of the present invention has been 
presented for purposes of illustration and description, 
but is not intended to be exhaustive or limited to the 
invention in the form disclosed. Many modifications and 
variations will be apparent to those of ordinary skill in 
the art. The embodiment was chosen and described in 
order to best explain the principles of the invention. 
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the practical application, and to enable others of 
ordinary sKill in the art to understand the invention 

V, ^- with various modifications as are 

various embodiments wirn vdxxu 

suited to the particular use contemplated. 



